Probabilistic Distance Measures in Reproducing Kernel Hibert Space

Probabilistic distance measures are important quantities in many research areas. For example, the Chernoff distance (or the Bhattacharyya distance as its special example) is often used to bound the Bayes error in a pattern classification task and the Kullback-Leibler (KL) distance is a key quantity in the information theory literature. However, computing these distances is a difficult task and analytic solutions are not available except under some special circumstances. One popular example is the Gaussian density. The Gaussian density employs only up to second-order statistics and hence is rather limited. In this paper, we enhance this capacity through a nonlinear mapping from original data space to reproducing kernel Hilbert space, which is implemented by a kernel embedding. Since this mapping is nonlinear, we present a new approach to study these distances whose feasibility and efficiency are demonstrated using experiments.